InfiniBand™ (IB) is a switched-fabric communications architecture that is widely used in high-performance computing. It has been standardized by the InfiniBand Trade Association. Computing devices (host processors and peripherals) connect to the IB fabric via a network interface controller (NIC), which is referred to in IB parlance as a channel adapter. Host processors (or hosts) use a host channel adapter (HCA), while peripheral devices use a target channel adapter (TCA).
Client processes running on a host processor, such as software application processes, communicate with the transport layer of the IB fabric by manipulating a transport service instance, known as a “queue pair” (QP), made up of a send work queue and a receive work queue (often referred to simply as a send queue and a receive queue). To send and receive messages over the network using a HCA, the client initiates work requests (WRs), which cause the HCA driver program to place work items, called work queue elements (WQEs), in the appropriate work queues. Normally, each WR has a data buffer associated with it, to be used for holding the data that is to be sent or received in executing the WQE. The HCA executes the WQEs and thus communicates with the corresponding QP of the channel adapter at the other end of the link. After it has finished servicing a WQE, the HCA typically writes a completion report, in the form of a completion queue element (CQE), to a completion queue, to be read by the client as an indication that the work request has been executed.
IB channel adapters implement various service types and transport operations, including remote direct memory access (RDMA) read and write operations, as well as send operations. Both RDMA write and send requests carry data sent by a channel adapter (known as the requester) and cause another channel adapter (the responder) to write the data to a memory address at its own end of the link. Whereas RDMA write requests specify the address in the remote responder's memory to which the data are to be written, send requests rely on the responder to determine the memory location at the request destination.
Although the above terminology and some of the embodiments in the description that follows are based on features of the IB architecture and use vocabulary taken from IB specifications, similar mechanisms exist in networks and I/O devices that operate in accordance with other protocols, such as Ethernet and Fibre Channel. The IB terminology and features are used herein by way of example, for the sake of convenience and clarity, and not by way of limitation.